Reliable and successful software is built through sound, efficient and thorough testing. However, software testing is labor intensive and expensive and accounts for a substantial portion of commercial software development costs. At the same time, software testing is critical and necessary to achieving quality software. Typically, software testing includes test suite generation, test suite execution validation, and regression testing.
Test suite generation involves creating a set of inputs which force the program or sub-program under test to execute different parts of the source code. This generated input set is called a “test suite.” A good test suite fully exercises the program's functionality including the individual functions, methods, classes, and the like.
Test suite execution validation involves executing the program using the test suite and evaluating the program's outputs to validate the program's correctness. Errors detected during this task are corrected and the process is repeated until all tests execute correctly. Regression testing involves verifying new versions of the program by running the program against the test suite and comparing its outputs to the expected outputs from the original program. This task ensures that changes in the program source code do not introduce new defects.
Historically, test suite generation, test suite execution validation and regression testing were manual tasks. Only efforts to automate regression testing have proven successful. Nevertheless, as the size and complexity of software programs increase, the need to automate test suite generation and test suite execution validation becomes more important. It is difficult to test all the methods and functions in a complex program and be able to isolate a single faulty function within the complex program.
However, the difficulty in generating a test suite that executes most of the source code increases as the number of functions that call other functions (nested functions) increases within the program under test. A significant number of functions need to be executed to force the program to test a single function that calls those significant number of functions. Furthermore, during development of a complex program by a team of programmers, functions and methods are developed (typically, by different individuals) that may call other parts of the program that has not been developed yet. To be able to thoroughly test these newly developed functions and methods, the programmers need to wait for the whole project to be completed.
A debugger may be used to step through the program under test to facilitate fixing of problems. However, the debugger only follows the path of execution. However, when the users need to go through small modules of the program, they need to define the path by specifying the functions to be called, the arguments, and the values to be returned.
Therefore, what is needed is an automated test tool for assisting programmers in testing software programs that is capable of covering all of the program under test by breaking down the program to smaller components and then test those smaller components individually. Furthermore, what is needed is a system and method for testing a class, function, routine, or method as soon as it is compiled without integrating it into a larger project.